Getting started

Loading essentia.js

• Using CDN

CDN links are available for serving essentia.js from jsDelivr and unpkg.

You can also find the latest releases on Github releases.

• HTML <script> tag

<script src="https://cdn.jsdelivr.net/npm/essentia.js@<version>/dist/essentia-wasm.web.js"></script>
<script src="https://cdn.jsdelivr.net/npm/essentia.js@<version>/dist/essentia.js-*.js"></script>

• ES6 style import

import Essentia from 'https://cdn.jsdelivr.net/npm/essentia.js@<version>/dist/essentia.js-core.es.js';
// import essentia-wasm-module
import { EssentiaWASM } from 'https://cdn.jsdelivr.net/npm/essentia.js@<version>/dist/essentia-wasm.es.js';
  
const essentia = new Essentia(EssentiaWASM);

// prints version of essentia wasm backend
console.log(essentia.version)

// prints all the available algorithm methods in Essentia
console.log(essentia.algorithmNames)

Note: You shouldn't import the essentia-wasm.es.js on the main thread. The ideal way is to use it along with the AudioWorklet design pattern or with WebWorkers. For async load of the library on main UI thread use essentia-wasm.web.js.

• NPM users

npm install essentia.js

For Yarn users

yarn add essentia.js

Usage example

let esPkg = require('essentia.js');

const essentia = new esPkg.Essentia(esPkg.EssentiaWASM);

// prints version of the essentia wasm backend
console.log(essentia.version)

// prints all the available algorithm methods in Essentia
console.log(essentia.algorithmNames)

Try it live on Runkit.

Usages in Javascript

After succesfully loading essentia.js-core** and essentia-wasm.** files on the JavaScript end, a typical usage would be like the following.

import Essentia from './essentia.js-core.es.js';
// import essentia-wasm-module
import { EssentiaWASM } from './essentia-wasm.es.js';

// create essentia object with all the methods to run various algorithms
// by loading the wasm back-end.
// here, `EssentiaModule` is an emscripten module object imported to the global namespace
let essentia = new Essentia(EssentiaWASM);

// run an specific algorithm
let yourOutput = essentia.'<your-essentia-algo>'(<inputs> ..., <parameters> (optional)...);

yourOutput.'<your_output-key>'

As you can see, in contrary to the essentia python bindings there are no seperate configure and compute methods for the algorithms in essentia.js since we abstract both into a single method.

For example, in real use-cases, it will look like below

const audioCtx = new AudioContext();

let audioURL = "https://freesound.org/data/previews/328/328857_230356-lq.mp3";

// decode audio data
const audioBuffer = await essentia.getAudioBufferFromURL(audioURL, audioCtx);            

/* OR
* you could also decode audio from any other 
* source and pass to an essentia algorithm. */

// convert the JS float32 typed array into std::vector<float>
const inputSignalVector = essentia.arrayToVector(audioBuffer.getChannelData(0));


// Computing ReplayGain from an input audio signal vector
// The algorithm return float type
// check https://essentia.upf.edu/reference/std_ReplayGain.html
let outputRG = essentia.ReplayGain(inputSignalVector, // input
                                    44100); // sampleRate (parameter optional)


console.log(outputRG.replayGain);

// Running PitchYinProbabilistic algorithm on an input audio signal vector
// check https://essentia.upf.edu/reference/std_PitchYinProbabilistic.html
let outputPyYin = essentia.PitchYinProbabilistic(inputSignalVector, // input
                                          // parameters (optional)
                                          4096, // frameSize 
                                          256, // hopSize
                                          0.1, // lowRMSThreshold
                                          'zero', // outputUnvoiced,
                                          false, // preciseTime
                                          44100); //sampleRate

let pitches = essentia.vectorToArray(outputPyYin.pitch);
let voicedProbabilities = essentia.vectorToArray(outputPyYin.voicedProbabilities);

console.log(pitch);
console.log(voicedProbabilities);

outputPyYin.pitch.delete()
outputPyYin.voicedProbabilities.delete()

// CAUTION: only use the `shutdown` and `delete` methods below if you've finished your analysis and don't plan on re-using Essentia again in your program lifecycle.

// call internal essentia::shutdown C++ method
essentia.shutdown();
// delete EssentiaJS instance, free JS memory 
essentia.delete();

Similarly, you could use any of the algorithms specified in the Essentia JS Core API.

Examples

Simple web example

<html lang="en">
  <head>
    <script src="https://cdn.jsdelivr.net/npm/essentia.js@<version>/dist/essentia-wasm.web.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/essentia.js@<version>/dist/essentia.js-core.js"></script>
    <script>
      let essentia;

      EssentiaWASM().then( function(EssentiaWasm) {
        essentia = new Essentia(EssentiaWasm);
        // prints version of the essentia wasm backend
        console.log(essentia.version)
        // prints all the available algorithms in essentia.js 
        console.log(essentia.algorithmNames);

        // add your custom audio feature extraction callbacks here
      });
    </script>
  </head>
</html>

NOTE: The above code pattern runs the process on the main UI thread. For costly algorithms, we recommend you to use AudioWorklets or use it inside a Web Worker instead.

Offline audio feature extraction

// generate overlapping frames of audio signal from given parameters
const frames = essentia.FrameGenerator(audioData, 
                                      1024, // frameSize
                                      512); // hopSize

// Iterate through every frame and do the desired audio processing
// In this case, we just apply a hanning window to the signal 
for (var i=0; i<frames.size(); i++) {
  // apply a window (hanning) to the signal using the default parameters
  let windowedOut = essentia.Windowing(frames.get(i));
  console.log(windowedOut.frame);

  // we can add any essentia algorithms here to the compute frame-wise audio feature
}